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Abstract 

Consider  a  feedforward  network  of  single-server  stations  populated  by  multiple  job  types. 
Each  job  requires  the  connpletion  of  a  number  of  tasks  whose  order  of  execution  is  determined 
by  a  set  of  deterministic  precedence  constraints.  The  precedence  requirements  allow  some  tasks 
to  be  done  in  parallel  (in  which  case  tasks  would  "fork")  and  require  that  others  be  processed 
sequentially  (where  tasks  may  "join").  Jobs  of  a  given  type  share  tiie  same  precedence  con- 
straints, interarrival  time  distributions,  and  service  time  distributions,  but  these  ciiaracteristics 
may  vary  across  different  job  types.  We  show  that  the  heavy  traffic  limit  of  certain  processes 
associated  with  heterogeneous  fork-join  networks  can  be  expressed  as  a  semimartingale  reflected 
Brownian  motion  with  polyhedral  state  space.  The  polyhedral  region  typically  has  many  more 
faces  than  its  dimension,  and  the  description  of  the  state  space  becomes  quite  complicated  in 
this  setting.  One  can  interpret  the  proliferation  of  additional  faces  in  heterogeneous  fork-join 
networks  as  (i)  articulations  of  the  fork  and  join  constraints,  and  (ii)  results  of  the  disordering 
effects  that  occur  when  jobs  fork  and  join  in  their  sojourns  through  the  network. 

KEYWORDS:  ioxV-]o\n  networks,  heterogeneous  customer  populations,  reflected  Brownian  motion. 
non-simple  polyhedral  state  space,  diffusion  approximations,  heavy  traffic  analysis. 
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1     Introduction  and  Summary 

We  consider  in  this  paper  the  class  of  feedforward  fork-join  networks  with  heterogeneous  customer 
populations.  The  network,  which  consists  of  d  single-server  stations,  is  populated  by  multiple  job 
types.  Each  job  requires  the  completion  of  a  number  of  tasks  whose  order  of  execution  is  determined 
by  a  set  of  deterministic  precedence  constraints.  The  precedence  requirements  allow  some  tasks 
to  be  performed  in  parallel  (in  which  case  tasks  would  fork)  and  require  that  others  be  processed 
sequentially  (where  tasks  may  join).  Jobs  of  a  given  type  share  the  same  precedence  constraints, 
interarrival  time  distributions,  and  service  time  distributions,  but  these  characteristics  may  vary 
across  different  job  types.  We  restrict  attention  to  the  case  where  the  network  is  feedforward:  that 
is,  stations  are  numbered  in  such  a  way  that  jobs  always  flow  from  lower  numbered  stations  to 
higher  numbered  ones. 

We  present  a  heavy  traffic  analysis  for  processing  networks  of  the  type  described  above.  It 
was  shown  in  Nguyen  [18]  that  when  the  customer  population  is  homogeneous  —  that  is,  when  all 
customers  share  the  same  precedence  requirements,  interarrival  time  distributions,  and  service  time 
distributions  —  the  heavy  traffic  behavior  of  the  network  can  be  approximated  by  a  d-dimensional 
semimartingale  reflected  Brownian  motion  (.SRBM)  whose  state  space  is  a  non-simple  convex  poly- 
hedral cone  in  the  nonnegative  orthant.  Unlike  the  corresponding  results  for  conventional  queueing 
networks  (networks  with  strictly  sequential  processing)  [11.  12,  13,  19,  20],  the  number  of  faces 
in  the  polyhedral  region  is  greater  than  d.  One  can  interpret  the  presence  of  additional  faces  as 
articulations  of  synchronization  constraints  embodied  in  the  fork  and  join  constructs. 

In  this  paper,  we  show  that  the  heavy  traffic  limit  of  certain  processes  associated  with  hetero- 
geneous fork-join  networks  can  also  be  expressed  as  c/-dimensional  SRBM's  with  polyhedral  state 
space.  However,  the  polyhedral  region  typically  has  many  more  faces  than  its  homogeneous  coun- 
terpart, and  the  description  of  the  state  space  becomes  vastly  more  complicated  in  this  setting. 
This  result  is  surprising  when  compared  to  those  a.ssociated  with  conventional  queueing  networks, 
where  the  form  of  the  limiting  process  does  not  change  with  the  presence  of  multiple  customer 
types  (this  is  a  result  of  the  "state-space  collapse"  phenomenon)  [19,  21].  One  can  interpret  the 
proliferation  of  additional  faces  in  heterogeneous  fork-join  networks  as  results  of  the  disordering 
effects  that  occur  when  jobs  fork  and  join  in  their  sojourns  through  the  network. 

Processing  systems  that  are  characterized  by  parallel  as  well  as  sequential  processing  exist  in 
many  industrial  settings.  Readers  may  refer  to  Baccelli  and  Makowski  [5],  Avi-Itzhak  [16],  and 
Nguyen  [18]  for  a  survey  of  several  interesting  applications.  The  generalization  of  [18]  to  allow 
multiple  job  types  constitutes  an  important  extension  from  the  practical  point  of  view.  Most  current 
treatises  of  fork-join  networks  assume  that  all  customers  are  statistically  similar  [5].  Baccelli  and 
Liu  [4]  consider  a  fork-join  network  in  which  a  job  may  send  batches  of  tasks  (that  may  include 
more  than  one  task)  to  processing  stations,  and  jobs  of  different  types  send  batches  of  different 
sizes.  Baccelli  and  Liu  still  assume,  however,  that  all  jobs  share  the  same  feedforward  deterministic 


routing  structure.  Such  a  model  can  represent,  for  example,  systems  in  which  some  processing 
stations  are  capable  of  performing  more  than  one  kind  of  task;  Baccelli  and  Liu  are  motivated  by 
multiprocessor  systems  running  parallel  programs. 

The  recent  works  by  Adler,  Mandelbaum,  Nguyen,  and  Schwerer  [1,  2.  3]  propose  a  processing 
network  model  for  studying  new  product  development.  The  model  they  describe,  which  they  simply 
call  a  "processing  network  model,"  is  more  encompassing  than  the  class  of  fork-join  networks 
studied  here.  The  key  restriction  in  this  paper,  which  is  not  assumed  in  [1,  2,  3].  is  that  jobs  must 
visit  workstations  in  a  feedforward  manner.  The  possibility  of  feedback  in  the  network  is  yet  an 
important  generalization  that  must  be  considered  in  future  work.  However,  as  the  work  by  Adler, 
Mandelbaum,  Nguyen,  and  Schwerer  demonstrates,  heterogeneity  in  the  customer  population  is  an 
essential  characteristic  that  must  be  captured. 

The  paper  is  organized  as  follows.  We  give  a  formal  description  of  the  model  in  Section  2  and 
define  the  processes  of  interest  Section  3.  In  order  to  state  the  heavy  traffic  limit  results  for  these 
processes,  one  must  refer  to  a  "sequence  of  systems."  Section  4  defines  such  a  sequence.  Before 
stating  the  heavy  traffic  limit  theorems,  we  introduce  some  additional  notation  and  preliminary 
results  in  Section  5.  The  main  theorems  are  then  summarized  in  Section  6,  where  we  also  illustrate 
the  heavy  traffic  limit  theorem  for  several  special  cases.  The  proof  of  these  theorems  are  then  given 
in  Section  7.  Finally,  some  concluding  remarks  are  given  in  Section  8. 

We  end  this  section  with  some  technical  preliminaries.  The  space  D'^[0,cc)  is  the  r-dimensional 
product  space  of  functions  /  :  [0,oc)  -~  W  that  are  right  continuous  on  [0,  >c)  and  have  left  limits 
on  (0,oo).  The  space  C^fO.oc)  is  endowed  with  the  Skorohod  topology  [6].  For  ,V"  a  sequence 
of  processes  in  D'"[0,oo)  and  A'  G  D''[0.oo),  we  write  A'"^=>A'  to  mean  A""  converges  to  X  in 
distribution. 

For  /  :  [0,.Do)  ^  >R,  set 

WfWt  =    sup    |/(.)|, 

0<J!<( 

and  for  a  vector- valued  function  /  =  (/i,  /2,  ■  ■    .  /r )'  :  [0,  oc)  —  9?'",  we  let 

|t   =   (ll/l||<,..-,||/r|lt)'. 


A  sequence  of  functions  {/"}  converges  to  a  function  /  uniformly  on  compact  .sets  (u.o.c.)  if  for 
each  t  >  0,  ||/"  -  f\\t  ^  0  as  n  —  oc.  For  a  sequence  of  functions  {A'"}  on  D'"[0,  >: )  and  A  a 
process  in  D'"[0,oo),  we  write  A'"  —  A'  u.o.c  if  almost  surely,  .V"  converges  to  A'  uniformly  on 
compact  sets. 

In  our  heavy  traffic  limit  theorems,  the  weak  limit  obtained  is  a  .semimartingale  reflected  Brow- 
nian  motion  whose  state  space  is  a  polyhedral  cone  in  the  nonnegative  orthant.  A  Brownian  motion 
process  having  drift  vector  d  and  covariance  matrix  P  will  be  denoted  as  (0,  r)BM;  likewise,  a  semi- 
martingale reflected  Brownian  motion  with  these  drift  and  covariance  parameters,  reflection  matrix 
R,  and  state  space  is  S  is  denoted  as  (5,  ^,  P, /?)SRBM. 
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TYPE  2 


Figure  1:  Tasks  and  Precedence  Constraints  of  Two  Job  Types 

2     Model  Description 

The  network  under  consideration  consists  of  d  single-server  stations  and  iiosts  p  types  of  jobs.  Jobs 
of  type  q  arrive  to  the  network  at  rate  a,.  Each  type  q  job  requires  completion  of  a  number  of 
activities.  Hereafter,  we  refer  to  a  job-type/activity  pair  as  a  task  or  a  class  interchangeably.  Task 
k  receives  service  from  station  _;'  =  s{k),  and  we  denote  by  r^  the  mean  service  time  for  task  k. 
Letting  Aq  denote  the  set  of  tasks  (or  classes)  in  job  type  q,  set 

A  =  AiU..  .UAp  -  {1 A'}. 

Our  convention  will  be  to  index  workstations  by  i.j  =  I. .  .  .  ,d.  job  types  by  q.r  —  1. .  ..  ,p,  and 
tasks  by  fc,/  =  1, .  .  .  ,  A'.  For  notational  convenience,  we  define  A^  =  a,  for  all  tasks  k  £  Aq  and  we 
write  q{k)  to  mean  the  job  type  q  for  which  k  £  Aq 

The  order  in  which  tasks  are  executed  is  determined  by  a  set  of  deterministic  precedence  con- 
straints, which  are  articulated  by  way  of  a  A'  x  A'  precedence  matrix  P  =  (Pyt/)  defined  as  follows: 


'it; 


_  j    1     if  task  k  is  an  immediate  predecessor  for  task  / 
0     otherwise. 


(2.1) 


(Because  all  elements  of  the  precedence  matrix  P  are  zeros  and  ones,  routing  of  tasks  is  clearly 
deterministic  within  each  job  type.)  We  assume  that  there  exists  a  column  and  row  permutation 
of  P  such  that  the  resulting  matrix  is  strictly  upper  triangular.  In  terms  of  the  model,  this  means 
that  each  tcisk  is  performed  exactlly  once  and  is  never  repeated.  From  the  precedence  matrix  P, 
we  can  now  define  the  set  of  immediate  predecessors  as 


V(i)   =   {keA  Pki  =  1}. 


(2.2) 


From  the  modelling  point  of  view,  V{1)  is  the  set  of  tasks  that  must  be  completed  before  task  /  can 
begin. 


Figure  2:  Tasks  and  Precedence  Constraints  of  Two  Job  Types 


For  the  two  job  types  depicted  in  Figure  1,  A\  =  {1,2,3,4},  ^2  =  {5.6.7},  'P(4)  =  {1.3}.  and 
the  precedence  matrix  P  is  given  by 

^0001000 
0  0  10  0  0  0 
0  0  0  10  0  0 
P=  0000000 
0  0  0  0  0  0  1 
0  0  0  0  0  0  1 
0  0  0  0  0  0  0 

We  allow  tasks  to  map  to  stations  in  a  many-one-to  fashion,  implying  that  while  each  station 
may  be  capable  of  performing  several  types  of  tasks,  each  task  is  performed  at  exactly  one  station. 
We  define  the  constitutency  of  station  /  =  1  .  .  .  ,  r/  as  the  set  of  tasks  tiiat  are  served  at  station  i: 


C(i)  =  {keA  -.sik)  =  /}. 


(2.3) 


We  write  c(i)  to  mean  the  cardinality  of  the  constituency  set  Cii).    Next,  define  the  predecessor 
and  successor  station  mappings  for  station  _;  =  1  .  .  .  ,  J  as  follows: 


Tr(j)     =      {i  =  s{k):h<£V{l),l€C{j)}^ 

a(j)   =    {i  =  s{k)  iev(k).tec(j)}. 


(2.4) 
(2.5) 


That  is,  k{j)  denotes  the  set  of  stations  whose  output  feeds  directly  into  station  j,  analogously, 
a{j)  is  the  set  of  stations  that  receive  input  from  station  j    In  conjuction  with  the  predecessor  and 


successor  station  mappings,  we  now  define  a  d  x  d  ■'routing'"  matrix  IP  =  IP,j  whose  elements  are 
given  by 

'    1     if/GT(j) 


IPu  =  i  ,  (2.6) 

0     otiierwise. 

We  assume  that  there  is  a  column  and  row  permutation  of  IP  such  that  the  resulting  matrix  is 
strictly  upper  triangular;  in  terms  of  the  model,  this  means  that  we  assume  jobs  traverse  the 
network  in  such  a  way  that  jobs  never  return  to  a  station  it  previously  visited.  In  addition,  this 
condition  implies  that  there  are  no  precedence  constraints  among  the  tasks  at  each  station.  This 
constitutes  the  feedforward  routing  assumption  stated  in  Section  1. 

It  will  be  useful  to  think  of  new  arrivals  to  the  network  as  originating  from  a  "dummy"  station 
0.  With  that  interpretation  in  mind  let  us  define  for  each  job  type  q 

A°     =     {i-G  A,  ;  P/t  =  Ofor  all  /e  A,}  (2,7) 

Al     =     {keA,:Pki  =  OkrMleA,}.  (2.8) 

Clearly,  processing  of  a  type  q  job  begins  with  those  tasks  k  G  .4°  and  ends  with  the  tasks  k  £  A^. 
Next,  set  V{k)  =  0  iiV{k)  =  0  (or  equivalently  if  t  G  >l°(<:)).  let  s(0)  =  0,  and  redefine  (2.4)-(2.5) 
accordingly.  Finally,  we  define 

<T(0)  =  {/:^(i)  =  {0}}. 

Thus,  the  stations  in  (t(0)  receive  only  external  arrivals  and  the  feedforward  assumption  guarantees 
that  cr(0)  ^  0.  Figure  2  shows  how  the  two  job  types  depicted  in  Figure  1  are  processed  in  a  network 
consisting  of  four  workstations.  For  this  example,  we  have  7r(  1 )  =  7r(2)  =  {0},  7r(3)  =  {0,2}, 
7r(4)  =  {1,3},  and  a{0)  =  {1,2}. 

A  node  j  is  said  to  be  a  fork  node  if  it  contains  a  task  k  G  C{j)  such  that  k  G  'P(0  for  more 
than  one  task  /  .  Similarly,  station  j  is  join  node  if 'P{k)  contains  more  than  one  element  for  some 
constituent  task  k  G  C{j).  At  a  join  node,  a  type  k  task  is  said  to  be  complete,  or  a  unit,  if  all  of 
its  predecessors  /  G  Vik)  have  completed  their  service. 

We  assume  that  tasks  are  served  at  each  station  in  a  FIFO  manner.  At  nodes  that  do  not 
involve  a  joining  of  tasks,  this  simply  means  that  the  tasks  are  served  in  the  order  of  their  arrival 
to  the  station.  At  join  nodes,  the  arrival  time  of  a  (complete)  task  is  defined  to  be  the  time  at 
which  its  last  predecessor  completes  service.  Note  that  such  a  service  discipline  considers  only 
local,  or  station-level,  information.  For  the  special  case  of  fork-join  networks  with  homogeneous 
customers,  we  argued  in  [18]  that  such  a  policy  is  equivalent  to  the  global  scheme  of  serving  tasks 
in  the  order  of  the  arrival  of  their  associated  jobs.  In  this  setting,  where  different  customer  types 
traverse  different  routes  through  the  network,  one  oiiserves  a  fundamentally  different  phenomenon. 
In  particular,  a  task  corresponding  to  a  later  arrival  may  enter  a  downstream  station  before  those 
tasks  associated  with  earlier  jobs.  For  example,  consider  the  scenario  depicted  in  Figure  3,  which 
shows  the  status  of  five  jobs  in  their  intermediate  stages  of  processing.  The  network  contains  three 


(SS) 


Order  of  arrivals:   D  A  (i  O  © 
Figure  3:  A  Fork-Join  Network  with  Jobs  in  Intermediate  Stages  of  Processing 

type  1  jobs  and  two  type  2  jobs,  which  arrived  to  the  network  in  the  order  1,  1,2,  1,2.  Each  job 
type  follows  the  routing  requirements  described  in  Figure  2.  We  ask  the  reader  to  focus  attention 
on  the  buffer  between  stations  3  and  4,  hereafter  referred  to  as  buffer  (3,4).  Note  that  the  first  type 
2  job  (the  third  job  to  arrive  to  the  network)  has  overtaken  the  first  two  arrivals  and  is  the  first 
job  to  reach  buffer  (3,4).  Moreover,  if  server  1  completes  his  next  task  before  server  3  can  finish 
his,  the  first  job  to  be  processed  by  server  4  will  be  the  type  2  job.  The  policy  of  serving  tasks  in 
the  order  of  their  arrival  to  a  station  may  therefore  result  in  serving  jobs  out  of  order.  Moreover, 
a  downstream  station  may  incur  delays  due  to  the  need  for  resequencing  tasks  (for  example,  if  the 
station  is  a  join  node)  that  were  overtaken  by  other  tasks  at  upstream  stations  (for  example,  due  to 
forks).  In  Figure  3,  for  instance,  server  4  must  remain  idle  even  though  each  of  his  incident  buffers 
is  nonempty.  In  this  paper  we  investigate  how  such  a  disordering  is  manifested  in  the  heavy  traffic 
limit. 

3     Representations  for  Processes  of  Interest 


To  construct  the  basic  stochastic  processes  associated  with  the  fork-join  network,  let  us  assume  a 
probability  space  (fi,  J",  P)  on  which  are  defined  .sequences  of  unitized  random  variables  {»,('), '  > 

1}  and  {vk(i),i  >  1},  q  —  l,...,p,  k  =  1 A',  where  u,(i)  and  i;jt(i)  are  strictly  positive  with 

unit  mean.  As  will  be  explained  in  the  next  section,  we  require  very  weak  assumptions  regarding 
the  joint  distribution  of  these  sequences  of  unitized  variables.  However,  readers  may  find  it  iielpful 
to  think  in  terms  of  the  concrete  case  where  each  is  a  sequence  of  i.i.d.    random  variables  and 


the  sequences  are  mutually  independent.  From  these  sequences,  the  interarrival  times  and  service 
times  for  the  network  are  constructed  by  setting  the  interarrival  time  for  the  /  job  of  type  q  to 
be  a~^Ug(i).  and  the  service  time  for  task  k  of  this  job  to  be  r;^- 1 7,- ( / ) .  Recall  that  q\j  is  the  average 
arrival  rate  for  new  type  q  jobs  and  r^.  is  the  mean  service  time  for  task  k.  Also  recall  that  A;-  =  a, 
for  each  k  E  Ag- 

To  construct  the  external  arrival  process  for  type  q  jobs,  set  Uq(0)  =  0  and  define 


Nq{t)  =  max{m  :  2J  a^  '"^(0  <  i} 


1=0 
For  k  =  1, ...  A',  let  \]:(t)  be  the  partial  sums  process  associated  with  the  service  times  for  tasks  k, 

[t] 

Next,  define  for  each  k  £  C{j)  and  j  =  1, . .  .  ,  d, 

L,Ut)  =  Vk(Ng^k)(t))  =  u-n-(  1)  +  ■  ■  ■  +  nv^,{^\^,^(f)),      k  e  C(j).  (3.1) 

The  process  Ljk(t)  is  called  the  class  k  total  workload  input  process  for  station  _;;  it  is  the  sum  of 
all  task  k  service  times  associated  with  those  jobs  that  enter  the  network  during  [0,/].  Note  that 
Ljk{t)  includes  service  times  corresponding  to  tasks  that  may  not  arrive  to  station  j  until  after 
time  t.  Set 

\keCu)  j 

because  t  is  the  potential  amount  of  work  that  can  be  processed  in  t  units  of  time,  ij{t)  is  the 
difference  between  the  workload  input  and  the  potential  workload  output,  and  for  this  reason  it  is 
called  the  total  workload  netflow  process  at  station  j. 

Let  us  choose  an  "external"  station  j  G  cr(0),  fixing  j  until  further  notice.  For  each  k  G  C{j), 
set  Ajk{t)  =  ^q{k){^)-  Next  let  Mjk(t)  =  Ljk{t)  and  A'j(/)  =  <,j(t).  Because  station  j  hosts  only 
external  arrivals,  Ajk(t)  is  the  number  of  task  k  that  has  actually  arrived  to  station  j  by  time  t. 
Similarly  Mjk(t)  is  the  amount  of  task  k  work  that  has  arrived  to  station  j  in  [0,/]  and  Xj[t)  is  the 
corresponding  immediate  workload  netflow  process  at  this  station. 

From  Section  2.2  of  [10],  we  can  verify  that  the  processes  Wj  and  Ij  are  uniquely  defined  by 
the  following  three  statements: 

Wj(t)  =  Xj(t)  +  !j{t)>0      for  all/ >  0;  (3.3) 

Ij{-)  is  continuous  and  nondecreasing  with  lj{0)  =  0;  (3.4) 

Ij(-)  increases  only  at  times  t  when  Wj(t)  =  0;  (3-5) 


moreover,  Ij  is  given  by  the  continuous  mapping 

/,(/)  =  -    'nf{-V,(s)}-.  (3.6) 

0<s<t 

One  interprets  Ij  as  the  cumulative  idleness  process  for  server  _;'  and  Wj  as  the  immediate  workload 
process  at  station  j.  That  is,  Wj{t)  corresponds  to  the  sum  of  the  impending  service  times  for 
(complete)  tasks  waiting  at  station  j  at  time  /,  plus  the  remaining  service  time  of  any  task  that 
may  be  in  service.  If  we  define  Zj[t)  to  be  the  total  amount  of  work  for  server  j  that  is  present 
anywhere  in  the  system  at  time  t,  then 

Zj(t)  =  ^j(t)  +  I,(t),  (3.7) 

and  Zj{t)  =  Wj{t)  as  a  consequence  of  j  G  o'(O).  Hereafter  we  refer  to  Zj{t)  as  the  total  workload 
process  for  server  _;'. 

Next,  let  r]j[t)  be  the  arrival  time  of  the  customer  in  service  at  station  j  at  time  t  if  Wj(t)  >  0. 
and  set  r]j{t)  =  t  otherwise,  letting  Yjkit)  be  the  amount  of  time  server  _;'  has  spent  serving  tasks 
k  in  [0,t],  it  follows  from  the  FIFO  service  discipline  at  each  station  that 

YMt)  =  MMnAi))  +  en{t),  (3.8) 

where  eijfe(0  's  the  amount  of  service  the  current  task  has  received  if  that  task  is  of  class  k  and 
€ik(t)  =  0  otherwise.  As  a  matter  of  definition,  qj(t)  is  bounded  by  the  immediate  workload  process 
as  follows: 

Wjirjjit))  <  t  -  n,{t)  <  \V,{r^J(t))  +  e2j{t).  (3.9) 

where  e2j(0  =  0  if  Wj{T]j(t))  =  0  and  otherwise  e2j(0  's  the  service  time  of  the  customer  currently 
occupying  server  j.  Letting  Sk  =  {Sk(i)J  >  0}  be  the  renewal  process  associated  with  the  task  k 
service  times  {T^Vkil),  TkVk(2), .  . .},  the  number  of  tasks  k  to  have  departed  from  station  j  by  time 
t,  denoted  as  Djk(t),  is  then  given  by 

Djkit)  =  Sk{Yjk(t)).  (3.10) 

Finally,  defining  Ujk(t)  to  be  the  the  total  amount  of  partial  work  associated  with  tasks  k  that  is 
present  anywhere  in  the  system  at  time  t,  it  follows  from  the  previous  definitions  that 

Ujk(t)  =  Ljk(t)-yjk{t).  (3.11) 

Moreover,  it  is  a  trivial  consequence  that  Zj{t)  =  II;.p(^'(  )  Ujk{t)- 

In  an  inductive  manner,  these  definitions  can  be  extended  to  all  stations  in  the  network.  Con- 
sider a  station  _;'  such  that  all  immediate  predeces.sor  stations  have  been  "treated",  that  is,  if  j  €  ^(j) 
then  for  each  /  e  C(j)  the  processes  A',(<),  W,(t).  l,{t).  Z,{t),  D,i{t),  and  U,t(t)  have  been  defined. 
For  each  task  k  £  C(j)  one  defines  its  arrival  process  to  be; 

A,k{t)=  {       ''^'  '''^'  (3  12) 

I    m\n,^p^^.^D,^l),(t)    otherwise. 


One  can  interpret  Aj^it)  as  the  number  of  complete  tasks  k  that  have  arrived  to  station  j  by  time  t. 
(We  take  the  convention  that  work  is  associated  with  complete  tasks,  so  incomplete  tasks  present 
no  work  to  the  server.)  The  immediate  workload  input  process  and  immediate  netflow  process  for 
station  j  are  defined,  respectively,  via 

Mjkit)  =  VkiAjkit))  =  rt  [(^.(1)  +  .  .  .  +  r;.(.4j;..(0)]  (3.13) 

and 


-v,(o  = 


(  \ 

\keC{j)  J 

The  workload  process  Wj,  the  idleness  process  Ij,  the  total  workload  process  Zj.  the  departure 
process  Djk,  and  the  partial  workload  process  Ujk  are  then  defined  exactly  as  in  (3.3)-(3.11).  The 
vector  processes  A'^,  V^.  L,  .4,  .U,  A',  W ,  /,  Z,  D,  and  U  are  then  defined  in  the  obvious  manner. 

The  throughput  time  of  a  job  is  the  length  of  time  between  the  job's  arrival  and  its  subsequent 
departure  from  the  system.  Let  Tg(t)  be  the  throughput  time  of  the  next  type  <]  job  to  enter  the 
network  after  time  t.  The  intermediate  process  Tfjf.(t),  k  6  Aq,  is  defined  to  be  the  "throughput 
time  through  task  t,"  which  is  the  amount  of  elapsed  time  until  task  k  is  completed.  As  a  matter 
of  definition,  we  have  the  relationship 

Tq{t)=  max{T,i(0}. 

To  define  the  intermediate  processes  Tgk(t),  we  first  define  for  each  job  type  q  the  process 

%{t)  =  a;'  u{l)  +  ■  ■  ■  +  a-'u(\{t))  +  a;'u{Nq(t)  +  I), 

interpreted  as  the  arrival  epoch  of  the  next  type  q  job  to  enter  the  network  after  time  t.  For  each 
task  k  G  A°,  let 

$,fc(f)     =     4>,(0      and 

Because  a  type  q  job  begins  immediately  with  tasks  k  G  >^°,  ^qk(i)  's  the  arrival  time  of  this  task 
to  station  s{k).  Furthermore,  because  tasks  are  served  in  a  first-in-first-out  manner,  the  amount 
of  time  this  task  must  spend  at  station  s{k)  is  precisely  the  amount  of  work  found  at  station  s(k) 
immediately  after  its  arrival  (which  includes  the  service  time  associated  with  the  new  arrival).  Thus 
Tqk{t)  is  the  total  sojourn  time  of  the  job  through  task  k. 

For  other  stations  in  the  network,  the  random  processes  4>,i.(0  and  T,a.(0  are  inductively  defined 
as  follows.  Suppose  that  k  is  a  task  such  that  T,/(/)  has  been  defined  for  each  /  6  V{k),  and  .set 

$,/,(«)     =     <!',(<)+   max  7^^,(0  (3.15) 

l€V[k) 

Tqkit)     =     ^nmxTqi{t)  +  W,(k^{^qk{t))-  (3-16) 


Recall  that  the  arrival  time  of  a  task  is  the  time  at  which  its  last  precedessor  task  is  completed.  (If 
task  it  requires  a  join,  there  could  be  a  gap  between  completion  times  of  its  multiple  predecessor 
tasks.)  Thus,  maXi^-p,i^.Tgi{t)  is  the  amount  of  time  that  elapses  until  task  k  •arrives"  at  station 
s(k),  ^qk{t)  is  precisely  its  time  of  arrival,  and  Tgk{t)  is  the  throughput  time  through  task  k. 

4     A  Sequence  of  Systems  in  Heavy  Traffic 

The  limit  theorems  stated  here  apply  to  systems  that  satisfy  conditions  of  "heavy  traffic."  For 
it  G  C{j),  let  pjk  —  AfcTt  be  the  workload  factor  at  station  j  associated  with  tasks  k,  and  define  the 
total  traffic  intensity  at  station  j  to  be 

keC(j)  keC(j] 

The  system  is  said  to  be  stable  if  pj  <  1  for  j'  =  1, . .  .  ,  J,  and  it  is  said  to  be  in  heavy  traffic  if  pj  is 
"approximately"  1  for  each  j.  The  precise  formulation  of  our  heavy  traffic  limit  theorem  requires 
the  construction  of  a  "sequence  of  systems,"  indexed  by  n,  whose  corresponding  traffic  intensities 
pj      converge  to  1  for  all  j. 

Recall  that  the  interarrival  times  and  service  times  for  the  network  are  defined  in  terms  of 
the  basic  sequences  of  unitized  random  variables  {uq(i)  :  /  >  1},  {('/;(/)  :  '  >  1},  9  =  1.  .  ,p, 
k  =  1,  . . .  ,  A'.  To  construct  a  sequence  of  fork-join  networks  we  further  require  sequences  of  positive 
constants  {oiq  ,  "  ^  l}i  {'"t  ,n>l},q  =  \,...,p,k  —  l,..,K.  In  the  n  system  of  the  sequence, 
the  interarrival  times  and  service  times  are  taken  to  be  Uq  (i)  =  ii,(!)/ag  and  i^"  (')  =  ^  ^fc(')' 
respectively.  For  the  n*^^  system,  a,  is  the  arrival  rate  of  type  q  jobs  and  r^."  is  the  mean  service 
time  for  task  k.  Setting  AJ."  =  a,  for  k  £  Aq,  define  the  traffic  intensities  p'"  as  in  (4.1)  using 
X]^    and  ^     in  place  of  Xk  and  Tj. 

The  convention  here  is  to  denote  a  parameter  or  a  process  a.ssociated  with  the  n"*  system  by 
the  superscript  "(")"•  For  example,  Nq  refers  to  the  external  arrival  process  for  type  7  jobs  in 
the  n     system.  Define  the  centered  processes 


ir\t) 

=   r,!"»(0-rr'< 

L^\t) 

^  i;i-'(o-p5>t 

M^it) 

=     A/*r'(0-p5'< 

Nt\t)     =     N^q''\t)  -  a^^h 
A%\t)     ^    A^\t)-X[-U 

sir\t)  =  5r'(0-(rrV'< 

The  results  in  this  paper  apply  to  processes  that  have  been  "scaled."  Let  A''"*  denote  a  "generic" 
process  associated  with  the  n""  system.  The  scaled  version  of  the  process  A''"',  denoted  as  A'",  is 
defined  via 

A"(0  =  n-'/^V<"'(n/). 

Hereafter,  when  we  say  a  "scaled"  process  and  write  the  process  with  a  superscript  "n  ".  we  mean 
a  process  whose  space  and  time  dimensions  have  been  scaled  in  the  manner  specified  above. 
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It  is  assumed  that  the  following  conditions  hold  for  the  uiput  processes  of  the  network.    First, 

fc      -  ^/c  and  r^. 


the  arrival  rates  and  mean  service  times  converge  to  finite  constants,  Aj.      — •  Xf.  and  n"    —  r^. 


In) 

k  =  1, .  .  .  ,  A'.  This  implies  that  p  -^  p^  —  Z];.pC(  )  '^*'"''*--  Furtiiermore,  it  is  assumed  that  there 
exists  a  d- vector  9  =  {dx ,  ...  ,64)  such  that  for  each  j  —  1,  ....  d,  — cc'  <  0j  <  >c  and 

ni/2(p(")_l)  — -Oj    as  n  -  oc.  (4.2) 

Condition  (4.2)  is  called  the  heavy  traffic  condition.  It  requires  not  only  that  p^  =  1  at  each  station, 
but  also  that  the  rate  of  convergence  is  "'sufficiently  fast"  and  is  uniform  for  all  stations.  Finally, 
it  is  assumed  that  there  is  a  d  x  d  covariance  matrix  Q  such  that  the  following  functional  central 
limit  theorem  holds  as  n  ^  cxd: 

(iV",l/",L")=>(,VM'*,L*),where  L'  is  a  (0,^)  Brownian  motion 

and  jV*,  V  are  also  Brownian  motions  with  zero  drift.  (4.3) 

To  explore  the  implications  and  restrictions  of  assumption  (4.3),  write  the  scaled  netflow  process 
(3.2)  as 

^;(<)-  E  i,\.«)  +  n'/'(p;"'-i)/. 

fceC'(0 
It  follows  from  (3.1)-(3.2)  and  a.ssumptions  (4.2),  (43)  that 

i;;-(0     =     V:{\ki)  +  TkN;^^p),      and  (4.4) 

Cj{t)     =       Yl    ^Jfc(0  +  ^j<.  (4.5) 

keCij) 

Defining  the  d  x  c  constituency  matrix  C  with  elements 

/  1   ifkec(i) 

C,k  =  S    ,,        ,  (4-6) 

(_   IJ     otherwise 

and  setting 

r  =  CQC,  (4.7) 

one  can  conclude  that 

(Ar",K",L",,e")^(^*>'*,L',r).   vvherer  is  (^.r)BM.  i^-^) 

Next,  recall  that  Sk  is  the  counting  process  associated  with  the  partial  sums  process  I4.  From 
Theorem  1  of  [14],  (4.3)  implies  that 

S]:^-T-^^h'^.  (4.9) 

Finally,  consider  the  special  case  in  which  {uq{i),i   >    1}  and   {vk{i},i  >    1},  are  mutually 
independent  sequences  of  i.i.d.  random  variables  such  that  Ug(i)  and  Vk{i)  have  squared  coefficients 
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of  variation  c^.  and  c^^^  respectively  (the  squared  coefficient  of  variation  of  a  random  variable  is 
defined  to  be  its  variance  divided  by  the  square  of  its  mean).  Then  A',"  is  a  renewal  process  with 
rate  A,  ,  and  a  simple  application  of  the  functional  central  limit  theorem  for  renewal  processes  [6] 
proves  that  Ng  =^N^,  where  A^^'  is  {0,  Xc^g)BM..  Because  L  ^  is  a  compound  renewal  process,  L" 
converges  to  (0,n)BM  by  Theorem  2.1  of  [23].  In  particular,  the  covariance  matrix  is  of  the  form 


Q 


u 


^fc^'(d-  +  <(t)) 

it  =  /, 

^^^^•'■'Ca,(fc) 

M/,    q{k)-- 

=  7(0, 

0 

otherwise. 

(4.10) 


It  is  not  necessary  to  assume  i.i.d.  sequences  for  the  convergence  in  (4.3)  to  hold.  See  Glynn  [9] 
for  examples  of  convergent  sequences  with  dependencies  and  non-stationarity. 

5     Additional  Notation  and  Preliniinaries 

We  devote  this  section  to  developing  some  additional  notation  and  preliminary  results  that  will  be 
useful  for  future  reference.  For  a  station  j  G  cr(0)  vve  define  its  "depth"  to  be  dfj)  =  0.  The  depths 
of  all  other  stations  are  then  defined  recursively  as; 

d(j)  =  l  +  max{d(0,/>  0,iG  tt{j)}  .  (51) 

It  follows  from  the  feedforward  structure  that  such  a  notion  of  depth  is  well  defined  and  that 
d{j)  <  d  for  all  stations  j  =  1, .  .  .  ,  d. 

In  Section  2  we  defined  Tr{i)  to  be  the  set  of  predecessor  stations  to  station  /.    It  will  also  be 
useful  to  define  the  set  of  tasks  preceding  station  i.   Recall  that  C(i)  is  the  constituency  of  station  i 

and  c{i)  is  the  cardinality  of  this  set.  Rather  than  labelling  tasks  in  C{i)  by  k  =  I /\'.  we  now 

enumerate  these  tasks  by  a\,..  .  ,a\-y  Set  T(0)  =  {0}  and  for  each  station  /  =  1 d.  define 

T{i)  =  {x  =  {xi x,^,))■.Xl£V(a])}.  (.5.2) 

Each  element  of  T(z)  is  a  vector  of  c(/)  components,  and  each  component  corresponds  to  a  prede- 
cessor task  of  a  constituent  task  in  station  i.  Set  T^{i)  =  T(i), 

J\i)  =  {(x\x')  :  x'  €  T{i)-x'  =  (X?,.  .  .,4.)).z/  e  7  (s{x]))}. 

and  define  T  (i)  recursively  as  follows: 

T''ii)  =  {x  =  (x' x"")  ;         x'  eT(i),xl  eT(s(xj^)) 


'^hh ^-1  ^ 


■^H-m;.......-,))}'         m 
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.1  \\.        .    ;.        -1  ./././'-2 


where  /i   =   1, .  .  .  ,  c(/);  I2   —   1 c(s(x'(  ));    .  .  .;  //,_i    =    1 c[s[x^~^     ^  _   )),  and  we  take  the 

convention  that  c(0)  =  0.  For  a  station  i  with  depth  greater  than  or  equal  to  h  and  x  ^'T  (/),  we 
define  the  set  of  indices 

C\x)     =     {/  =  (/, lk)-h  =  l c(0;/2  =  l c(s(r,\))  :    ...; 

In  addition,  let 

r^'-tx)  =  {/  =  (/i, ...,/,)  e  £''(x) :  s(xf, i^ )  =  j}.  (5.5) 

Taking  the  network  in  Figure  2  as  an  example,  we  have  d(4)  =  2  and  T  (4)  =  (w.x.y,:)  where 


w^     =     (1,5)                  w'i     =     (0,0)  u^^  =  (0,0) 

x^     =     (1,6)                  2-?     =     (2,0)  x]  =  (0,0) 

y'     =     (3,5)                  y?     =     (0,0)  y|  =  (2.0) 

zi     =     (3,6)                  z?     =     (2,0)  -I  =  (2,0) 

For  the  moment,  consider  x  £  T  (4),  for  which  we  have 


(5.6) 


£2(1.)     =     {(1,1), (1,2), (2,1), (2,2)} 
Cjix)     =     {(1,1)}. 

One  may  think  of  each  element  x  £T  ''*(/')  as  describing  a  "path"  of  tasks  traversed  by  the  various 
job  types  on  their  sojourns  towards  station  )'.  Moreover,  Cj(x)  identifies  the  particular  "branch(es)" 
in  the  path  i  that  would  include  a  visit  to  station  _;'.  Using  the  notation  established  above,  the 
following  lemma  can  be  verified  directly. 

Lemma  5.1    For  each  station  i  =,.  .  .  ,d  and  6/^;  a  sequence  of  numbers  associated  with  tasks  kj, 


=    max         ^    b^. 


V  max  bki 

6     The  Ma,in  Results 
Theorem  6.1  Suppose  thai  assumptions  (^.2)  and  (^.3)  hold.   Then 

(C  ir ,  z" ,  M/" ,  r  )=>[C ,  v ,  z' ,  w ,  r ), 

where  for  each  i  =  I, .  . .  ,d  and  k  G  C{i):  U^  =  0, 

^''  is  a  {d,r)  Browman  motion;  (61) 
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'-  s{l)l 


U'k  =  P,k 


Z'  +    max 


max 


U' 


teV(k)  Ps{l)i  ^n,  .  \neP[m)  P,[n)n 

/*  is  continuous  and  non decreasing  with  /'(O)  —  0," 
/'   increases  only  at  times  t  with  W'(t)  =  0. 


(6.3) 

(6.4) 

(6.5) 

(6.6) 
(6.7) 


Set 

0  if  i-  =  0  or  /  =  0,  otherwise 

7fc/  =  I    l-Ps(k)k-      'fi--/.  (6.8) 

-Ps{t)h  if  i-  1^  I 

For  notational  convenience,  we  henceforth  write  s'"  to  mean  six]"-     ,    ).   Denoting  bv  h  the  depth 
of  a  station  i,  define  for  each  x  £  T  (i)  and  ji'  =  1  . .  .  ,  d  the  following  factors: 


+ 


E 


(6.9) 


We  then  define  the  convex  polyhedral  cone  S  to  be 

d 


^  =  0    n    <;  --  =  ( '1 ,  ■  ■  ■ ,  ^d)'  ■■  -'.  -  E  ^u(-^-)-i  ^  0 


(6.10) 


lev      (i) 
It  is  eeisily  verified  that  2  >  0  if  r  €  5.  For  each  /  =  1,  . . .  ,  cf,  we  also  define  the  boundary  set 

d  1 


Theorem  6.2   For  each  i  —  1 d, 


F'=        (J  ce5:r,  -^/i„(x)r,  =0 

d(.).  .  I  J  =  i 


(6  II) 


^'  is  a  (5,  F)  Brownian  motion; 


W:  =  Z:-      max       \'^l3„(x)Z']; 

/*  is  continuous  and  nondecreasmg  with  /,'(0)  =  0.' 
/'  increases  only  at  times  t  with  Z'{t)  G  F' . 


(6.12) 
(6.13) 

(6.14) 

(6.15) 
(6.16) 
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That  IS,  the  process  Z'  as  defined  in  Theorem  6  1  is  a  d-diinensional  SRBM  whose  state  space  is 
a  convex  polyhedral  cone  S-  The  SRBM  Z'  has  drift  6,  covariance  matrix  F,  and  reflection  matrix 
R  =  I  where  I  is  the  d-dimensional  identity  matrix. 


Remark:     The  results  of  Nguyen  [17]  may  be  applied  to  show  that  such  an  SRBM  is  well  defined 
in  the  strong  sense. 

To  interpret  Theorem  6.2,  note  that  statements  (6.13)  and  (6. 15)  are  reiterations  of  the  charac- 
terizations given  in  (37)  and  (3.4),  respectively.  Moreover,  the  approximation  (6  12)  of  the  netflow 
process  by  a  Brownian  motion  was  justified  in  Section  4  under  assumptions  (4.2)  and  (4.3).  Next, 
recall  that  each  element  x  £  T  '''(/)  describes  a  ''path"  of  tasks  traversed  by  the  various  con- 
stituent jobs  on  their  way  to  station  /.  Equation  (6. 15)  states  that  the  immediate  workload  at 
station  i  is  the  minimum  amount  of  work  found  among  all  the  "paths"  leading  up  to  station  i. 
In  other  words,  (6.15)  articulate  the  constraint  that  a  task  at  station  i  cannot  be  processed  until 
all  of  its  precedessor  tasks  have  been  completed  (this  is  the  definition  of  a  join  node).  With  this 
interpretation  in  mind,  statement  (6.16)  is  then  equivalent  to  (35).  Thus,  each  idleness  process 
is  associated  with  potentially  multiple  boundaries  on  the  state  space  5.  In  Nguyen  [18],  it  was 
argued  that  the  additional  faces  correspond  to  the  fork  and  join  constraints  in  the  network.  As  we 
will  demonstrate  in  an  example,  the  polyhedral  state  space  associated  with  heterogeneous  fork-join 
networks  typically  has  many  more  faces  than  its  homogeneous  counterpart.  These  additional  faces 
may  be  interpreted  as  results  of  the  disordering  effects  that  occur  when  jobs  fork  and  join  in  their 
sojourns  through  the  network. 

Example  1:  The  Sample  Fork- Join  Network 

The  heavy  traffic  limit  of  the  network  pictured  in  Figure  2  is  given  by 
4'  is  a  (^,r)  Brownian  motion; 

/*  is  continuous  and  nondecreasing  with  /*(0)  =  0; 

/*  increases  only  at  times  t  with  Z'{t)  =  0,  /  =  1,2; 

I^  increases  only  at  times  t  with  Z^{t)  —  p33Z2{t)  —  0; 

I^  increases  only  at  times  t  such  that  one  of  the  following  conditions  hold: 

Z;it)  -  Zj  (0  =  0,  OR 

Z;{t)  -  (p^,p:ie  -  P47P33)Z2it)  -  Zlit)  =  0,  OR 

Zlit)  -  P44Z:(t)  +  P47P33ZUt)  -  P47Z^(t)  =  0,    OR 
^4(0  -  P47Z'(i)  -  P44P36Z2{t)  -  P44^3(0  =  0. 
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6.17) 

6.18) 

6.19) 

6.20) 

6.21) 

6.22) 

6.23) 

6.24) 

6.25) 

Hence,  setting 


.4  = 


1 

0 

0 

0 

0 

1 

0 

0 

0 

-P33 

1 

0 

-1 

0 

0 

1 

0 

-  (P4-1P36  -  Pa- 

•P33) 

-1 

1 

'P-14 

P47P33 

-P47 

1 

-P47 

-P4AP36 

-PAA 

1 

and  letting  S  =  {x  :  Ax  >  0},  Z'  is  a  4-dimeiisional  SRBM  with  drift  6.  covariance  matrix  T, 
reflection  matrix  R  —  U  and  whose  state  space  is  the  convex  polyhedral  cone  S  with  seven  faces. 
We  now  turn  to  the  interpretation  of  conditions  (6.20)-(6.25).  Because  the  immediate  workload  is 
identical  to  the  total  workload  process  at  stations  1  and  2,  condition  (6. 20)  states  that  the  idleness 
processes  at  these  stations  increases  only  when  the  immediate  workload  at  the  respective  stations 
is  zero.  Emulating  the  arguments  in  Harrison  and  Nguyen  [12,  13],  one  may  think  of  \2Z2(t)  as 
the  number  of  tasks  2  occupying  station  2  at  time  t.  Consequently,  7^X221(1)  =  pzzZl(t)  is  the 
amount  of  work  cissociated  with  task  3  destined  for  station  3  that  still  reside  at  station  2  at  time  t. 
Hence  Wl{t)  =  Zl[i)  -  pj3Z2{i)  and  condition  (621)  specifies  that  the  idleness  process  at  station 
3  increases  only  when  there  is  no  immediate  work  at  that  station. 

Conditions  (6.22)-(6.25)  describe  the  four  scenarios  under  which  server  4  is  forced  to  remain  idle. 
It  can  be  verified  that  conditions  (6.22),  (6.23),  (6. 24),  (6.25)  correspond  to  the  paths  described 
by  w,  2,  y,  and  x  of  equation  (5.6),  respectively.  The  first  path,  w,  contains  the  predecessor  tasks 

1  and  5,  hence  condition  (6.22)  states  that  the  total  amount  of  work  destined  for  station  4  is 
contained  in  those  tasks  currently  waiting  for  service  at  station  1  either  in  the  form  of  task  1  or 
task  5.  That  is,  the  immediate  work  content  in  buffer  (1,4)  (the  buffer  joinmg  stations  1  and  4), 
given  by  Z^[t)  —  Z^(t),  is  zero.  Path  ;,  on  the  other  hand,  contains  the  predecessor  tasks  3  and  6, 
and  one  can  apply  a  similar  argument  to  arrive  at  the  conclusion  that  condition  (6. 23)  corresponds 
to  the  scenario  in  which  buffer  (3,4)  is  empty.  That  is,  the  immediate  work  content  in  buffer  (3,4) 
is  given  by  Z^{t)  -  (pAApje  -  PaiPss)  2^2(0  ~  •^3(0-  Because  station  4  is  a  join  node  for  both  types 
of  customers,  it  seems  clear  that  both  buffers  (2,4)  (the  buffer  joining  stations  2  and  4)  and  (3,4) 
(the  buffer  joining  stations  3  and  4)  must  be  nonempty  if  server  4  is  to  remain  busy.  However, 
these  are  not  the  only  times  at  which  server  4  may  be  idle.  Condition  (6.24),  which  corresponds 
to  path  I,  states  that  the  total  work  destined  for  station  4  are  completely  contained  in  tasks  3  at 
station  2  and  tasks  5  at  station  1.   That  is,  buffer  (1,4)  contains  no  tasks  corresponding  to  type 

2  jobs  and  buffer  (3,4)  does  not  have  any  type  1  work.  Because  there  are  no  complete  tasks  for 
station  4,  the  server  is  forced  to  remain  idle  although  both  of  its  incident  buffers  may  be  nonempty. 
Similarly,  condition  (625)  describes  the  converse  situation  in  which  buffer  (1,4)  has  only  type  2 
tasks  and  buffer  (3,  4)  contains  only  type  1  tasks.  I 
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Example  2:   Multiple  Customer  Types  with  Common  Routing  Structure 

Consider  a  fork-join  network  in  which  all  customers  share  the  same  routing  constraints,  but  the 
interarrival  times  and  service  times  may  be  different  across  job  types.  This  class  of  networks 
correspond  to  a  particularly  simple  case  of  the  networks  considered  in  this  paper.  For  stations 
j  G  cr(0),  it  follows  from  (6.5)  that  U'f.  =  pikZ* -  By  induction  on  the  depth  of  stations,  one  can 
verify  that  equation  (6  5)  becomes 

U'k  -  PikZ' +  Ptk   niax   2",,,  - /3,fc     51     ^""    "iP'    ^'m-  (6-26) 

meC(i) 

Because  all  customer  types  share  the  same  the  same  routing  structure,  the  sets  {s[n)  :  n  G  V(rn)} 
are  identical  for  each  task  m  G  C{i)-  Hence  the  last  term  on  the  right  side  of  (6.26)  is  equal  to 


-Pik 


and  equation  (6.26)  reduces  to 


/ 

\meC{i)j 


p,m  max  Z*  =  -p,k  max  Z' 


U:k  =  P.kZ: 


Hence,  for  z  =  1, .  .  .  ,  d  and  k  G  C{i), 

^*  is  a  (0,  F)  Brownian  motion; 

W'  =  Z'  —  max  Z'; 

U'k  =  PikZ'\ 

/'  is  continuous  and  nondecreasing  with  /'(O)  =  0; 

/'  increases  only  at  times  t  with  [V'(t)  =  0. 

Fork-join  networks  with  one  customer  type  (that  is,  homogeneous  fork-join  networks)  are  clearly 
a  special  subset  of  the  networks  discussed  in  this  section.  It  is  straightforward  to  verify  that  the 
results  above  agree  with  those  given  in  Nguyen  [18].  I 

Example  3:  Feedforward  Multi-Class  Queueing  Networks 

Consider  now  the  feedforward  multi-class  queueing  network  studied  by  Peterson  [19].  The  networks 
described  in  Peterson  [19]  are  essentially  similar  to  those  considered  here  with  one  important 
exception:  The  networks  in  [19]  require  that  Vlk)  contains  at  most  one  element  for  each  task  k\ 
that  is,  there  are  no  join  nodes.    (Peterson's  work  does  not  explicitly  consider  the  case  in  which 


17 


tasks  may  fork,  but  the  inclusion  of  the  forking  structure  would  not  pose  much  hardship  to  his 
analysis.)  Recall  that  Z'{t)  =  YlkeCii)  '-''fcO-  hence  equations  (6.2)-(6.5)  imply 


Pik 


W'  +    max 


^%)i 


leV(k)  Ps(i)i 


z'  =  \v:{t)+  Y.  "^^-^  p'^ 


k^C[x) 


f'Tt/,/ 


i^Vik)  Ps(l)l 


(6.27) 
(6.28) 


We  denote  by  p(/t)  the  one  predecessor  task  of  task  k,  and  efine  p(0)  =  0.  Setting  p^(k)  =  p(t). 
we  recursively  define  p^{k)  =  p(p''-i(fc)).  Letting  Uqq  =  IV'J  =  0,  equations  (6.27)-(6.28)  thus 
reduce  to 


u:. 


Pxk 


W'  +  ^^'<P'"^))P'(^) 


Ps(p'{k)]p'(k) 


z:  =  w:{t)+  Y.  ^- 


^' s(pHk))p^  ik) 


Similarly  to  the  previous  example,  we  can  use  induction  to  show  that  for  a  station  ;'  of  depth  /i, 

h 

9  =  1 

z:  =  w:{t)  +  Yl  E  M-in'(p«(.))- 

Readers  can  verify  that  this  agrees  with  the  result  obtained  by  Peterson  [19].  I 

Theorem  6.3    Under  assumptions  (4.2)  and  (4.3),  {T^, . . .  ,T^)=^(T{ , . . .  .Tp)  where 

r;(<)   =    maxr;,(<), 

keAl 

T;,{t)     =      max  T,'(0  +  tV-,),    T;o(/)  =  0. 
ieV{k) 

If  we  denote  by  /,  the  PERT/CPM  "longest  path  operator'"  for  type  q  jobs.  Theorem  6  3  implies 
the  representation 

^;(0  =  ',(w;,,),teA)-  (6-29) 

As  discussed  in  Nguyen  [18],  expression  (6. 29)  is  an  example  of  Reiman's  "snapshot"  principle  [20]. 
That  is,  in  the  heavy  traffic  scaling,  the  fluctuation  in  workload  levels  is  insignificantly  small  relative 
to  the  length  of  time  that  a  job  spends  in  the  system,  hence  a  "snapshot"  of  the  system  at  the  time 
of  a  job's  arrival  remains  representative  throughout  the  job's  sojourn  in  the  network.  Equation 
(6.29)  expresses  the  remarkable  result  that  sojourn  time  analysis  of  a  fork-join  network  may  be 
phrjised  in  terms  of  the  familiar  longest  path  analysis  of  PERT/CPM  methods,  where  traditional 
task  times  are  now  replaced  by  waiting  times  at  stations  corresponding  to  the  tasks. 
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Proofs 


By  Skorohod's  representation  theorem  and  the  continuity  of  Brownian  motions,  we  can  and  will 
assume  that  the  convergence  in  (4.3)  is  almost  surely  uniform  on  compact  sets:  that  is,  we  henceforth 
assume 

{N',V',L')  -  (N',V',r)  u.o.c.  (7.1) 

We  begin  the  proof  of  Theorem  6.1  with  a  few  preliminary  results.  The  first  lemma  is  an  immediate 
consequence  of  assumptions  (4.2)  and  (7.1). 

Lemma  7.1  ^"  ^  ^'  u.o.c.  where  ^'  is  a  (^,  F)  Brownian  motion  where  F  =  CClC ■ 

Lemma  7.2    For  fc  =  1, .  .  . ,  A',  j  =  1 d,  let  €^i^{t)  =  n-^^^e['l\nt)  and  €^j{t)  =  n-^/^€^^]\nt). 

Then  f"^  — «•  0  and  €2,  ~^  0  u.o.c. 


Proof.     Note  that 


and 


0<fll.'(O<        max        r<"'ufc(/) 


0<  4'^*(*)  <    max         max        r|."'(.;i(;) 


The  lemma  follows  directly  from  assumption  (7.1)  and  Lemma  3.3  of  Iglehart  and  Whitt  [15].      I 

Let  r7j(f)  =  ^-^'"'("0  and  r]'^(t)  =  n'^l'^int  -  rjj"'(nf)). 

Lemma  7.3   If  Wf  —  W*  u.o.c,  then  rj"  —  e  u.o.c   where  e(i)  =  /. 

Proof.     Because  r7"(f)  <  t,  it  follows  from  equation  (3.9)  that  for  each  /  >  0, 

l|e(-)-'7;(-)||*     <     n-'/'||tV;(r?-(-))l|t  +  "-'/'lk5(')llt 
<     n-i/2||VV7(.)||,  +  n-'/'||£5(.)||t. 

From  Lemma  7.2  and  the  assumption  that  l-V'"  —  W'  u.o.c,  it  follows  that  ||e()  -  »7"()||(  ^  0.  I 
Lemma  7.4   If  W^  —  W'  u.o.c,  then  r]^  --  Wj'  u.o.c. 

Proof.     It  follows  from  equation  (3.9)  that  for  each  t  >  0, 

\\ri;(')  -  w;n\\t  <  \\w;{-)  -  py;(-)||«  +  W^(n^{-))  -  K(-)\\t  +  Mi-)\\t-  (7.2) 
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As  a  result  of  Lemma  7.3  and  the  continuity  of  W' ,  the  first  two  terms  on  the  right  side  of  (7.2) 
converges  to  zero.  Invoking  Lemma  7  2,  one  concludes  that  rj^  -~  IVJ  u.o.c.  I 

The  proof  of  Theorem  6.1  proceeds  by  induction  on  the  depth  of  stations.    \Ve  begin  with  the 
following  result  for  stations  of  depth  0. 

Lemma  7.5    Theorem  6.1  holds  for  all  stations  of  depth  0.  namely,  stations  j  £  cr{0). 


Proof.  Note  that  V(k)  =  0  for  each  task  k  £  C(j)  when  j  6  cr(0).  Because  A'^"  =  ^^  for 
j  G  ''■(0),  it  follows  from  Lemma  7.1,  equations  (3.3),  (3.6),  (3.7),  and  the  continuous  mapping 
theorem  that  {Xf,  W]",  q,  Z^)  —  (X',  WJ,I^,ZJ)  u.o.c.  where  XJ  =  ^;,  i;{t)  =  -  info<,<(  A';(s), 
W;{t)  =  X;{t)  +  /;(0,  and  Z;(0  =  ^;(t)  +  I-{t).  Because 

U^,[t)  =  L^,{t)  -  L;,(»7;«))  +  P%^r,j(t)  -  e^i)^ 

it  follows  from  Lemma  7.2,  Lemma  7.4,  and  the  continuity  of  L'f.  that  ('J^.  —  U').  u.o.c.  where 
U\{t)  =  pjk^'{t)  =  pjkZ'At).  Joint  convergence  of  the  processes  of  interest  is  a  natural  conse- 
quence of  their  continuity.  I 


Proof  of  Theorem  6.1:  Define 

Y^l(t)  =  n-i/2(r;;'(n0-p;:'0    and    y;,(0  =  r,-iy;';*(n<), 
and  note  that  as  a  consequence  of  (3.11), 

y,",(0  =  L,",(0  -  (7,(0.  (7.3) 

With  Lemma  7.5  we  may  assume  inductively  that  the  convergence  in  Theorem  61  has  been  estab- 
lished for  all  stations  with  depth  h  or  less.  Consider  a  station  j  with  d(j)  =  h  -\-  \.  For  each  task 
it  G  C(j),  it  follows  from  (3.12)  that 


AU^)  =  { 


Mfc)(t)  if^(t)  =  0, 

^"/eT'tfc)  \s:(,),iy:(,),it))  +  ^;on(/)/(o|     otherwise. 


(Note  that  A*;  =  A,  for  all  /  G  Vik).)  Setting  ^"^.(t)  =  n-M^2*("0.  equation  (3. 13)  gives 
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Because  d(s(/))  <  h  for  all  tasks  /  G  V(k),  it  follows  from  (73)  and  the  induction  hypothesis  that 
^s(l)i  -^  K{l)l  "oc-  where  V;,,),!*)  =  Ll^yit)  -  t7,,);(i).  and  YJ^^^^  -  p,,,,,  ■  e  u.o.c.  where  that 
e{t)  =  t.  Consequently,  if  V{lc)  ^  0,  M^f.  —  MJf.  u.o.c.  with 


M 


■;,(t)     =     K;(A,.0+r,    mm    {-i",-'^' V',' (P.(0/0  +  ^r' i:(/)/(0  -  rr^f'^OiiO} 


;e7'(Jt) 


n*(^fcO+  ^fc    nim    {-r-H7(A,/)  +  rf^  (r,'(A,/)  +  rjAT*;)!/))  -  ^r' ^ 's'/He)} 


=     V:{Xkt)  +  ^fciVV)(f)  -  Tfc   nnax   ^^n^,  (7.4) 

where  the  last  equality  follows  because  q(k)  —  q(l)  for  /  G  Vik).   On  the  other  hand,  if  ■P(A-)  =  0. 
we  have  M"^  — *  M*;.  u.o.c.  with 

M;;  =  V';(A,/)+rtA-,)(f). 
Applying  the  continuous  mapping  theorem  to  equation  (3.14),  A'"  —  A'*  u.o.c.  where 

^'(i)    =     E   (^fc'(^/cO  +  r,A^;,)(o)-   E    %a-^  7f';m/(^)  +  ^/ 


=    Cj{^)  -   E  ^j*^  "^-'*^ 


and  we  use  the  convention  that  max  0  =  0.  That  (WJ'.P.Z")  —  (\V' ,  I' .  Z')  u.o.c.  is  again  a 
consequence  of  the  continuous  mapping  theorem  by  virtue  of  equations  (33),  (3.6),  and  (3.7).  All 
that  remains  is  to  prove  the  convergence  of  U^ .  From  Lemma  73  and  Lemma  7.4,  we  have  17"  -~  e 
u.o.c.  and  t)"  —  W*  u.o.c.  Consequently,  it  follows  from  (3.8)  that 

YJ'it)  =  M;,(;7;(0)  -  P%''v]{t)  +  e-,,{t), 
from  which  we  can  conclude  V"  —  Y*  u.o.c.  where 

Y;(t)  =  M;,(t)-pjkW;{t).  (7.5) 

Because  U^^i^)  =  L'Jf.{t)  -  YJl{t),  it  follows  from  (7.4)  and  (7.5)  that  [/J\.  —  U'^.  u.o.c.  and 

u;,{t)  =   L',(t)-T;,[t) 

=     L]f,(t)  -  (v;r(A,0  +  rkN'^,^{t))  +  Tk  max   -^^^^^  +  PjkWJit) 
=     PjkZ  (t)  +  Pjk    max pjk      >        Pjm     max     . 

leV{k)       Ps(l)l  p,    .  neV(m)       Ps{n)n 
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Proof  of  Theorenj  6.2:  Define  fki  as  in  (6.8),  and  rewrite  equation  (6.5)  as 


U'k  =  P,k 


rym       ,  V^  I  s{n)n 

Z,  +     2^     1km  [     max    

^p,   .  \yieP(m)   Ps{n)n 

met(i) 


7.6) 


Recall  that  a],  I  =  1.  .  .  .  .c(i).  enumerate  the  elements  of  C{i).    Applying  Lemma  5.1  to  (76).  we 
obtain 

U:,  =  p^k    Z:+   max    1^^      -ii£iifi|     .  (7.7) 


X6T(0     \    ;^j  '    Ps{x,)T, 


Similarly,  equation  (6.3)  is  equivalent  to  the  following  expression  as  a  result  of  Lemma  5.1, 

A,   =^,   -    max       l^P^a'.- ■ 

r€7(.)   \^,=  l         '  Ps{iu)i-i  J 

Substituting  (6.2)  and  (6.3)  in  (6.4)  and  applying  (7.8),  we  have 


(7.8) 


W'  =  Z'  -    max      ^  p,^. 


s(xt)n 


Substituting  (7.7)  in  the  above  expression,  we  obtain 

W'     =     Z'  —    max        >     p,„i      Z',  i  ,  +        max  >        7       „  ^1 


c(.) 


=   z- 


max 


r=(xl,x2)eT   ( 


c(.)  "^(^(^Ii 


u:,. 


E     ^.a;7       .u;.;'^"'-'^'"M(7.9) 


For  notational  convenience,  we  henceforth  write  s"*  to  mean  s{x"^  ^    ).    Substituting  (7.7)  in 
(7.9)  recursively,  one  can  verify  that  for  a  station  of  depth  h 


c(.) 


c(0  c(.') 


(»•)  cis"-') 

£•■■    E    P< T.    ,.  ••■7,h-, 


w-    =    Z-- 

x€7 
c(.)  cis"-') 


z;,  +  •  •  • 


^-1  Z'/, 


7.10) 


It  is  straightforward  to  verify  that  (7.10)  is  equivalent  to  equation  (615)  and  the  theorem  is  thus 
proved.  I 
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Remark:     Substituting  (6.2)-(6.4)  in  equation  (6.5),  we  obtain 


U'k  =  Pxk 


W'  +    max 


'-s{l]l 


leV(k)  Psii)t 


Because  Z'{t)  =  YikeCu)  ^^'fcC)'  '^  follows  that 


max  p,k- 


'-  s{l)l 


ceCo'^'^*''       /'.(O' 


7.11) 


Proceeding  in  the  same  manner  as  in  the  proof  of  Theorem  6.2,  we  can  show  that  for  a  station  ;'  of 
depth  /i,  (7.11)  can  be  written  as 


z:  =  w:  +  max     J2  Pxai  K^  +  ■■■+  E  Pxai  fv-;. 


(7.12) 


Readers  may  recognize  that  equation  (7.12)  is  an  "inverse"  formulation  of  the  relationship  described 
in  (6.15).  It  states  that  the  amount  of  total  work  in  the  system  for  station  i  is  the  maximum  of  the 
amount  of  immediate  work  destined  for  station  i  found  along  each  path  to  that  station. 


-i^(")/ 


-i^(")/ 


Proof  of  Theorem  6.3:   Define  <I>^(0  =  n'^^  '{nt)  and  <I>^;.(<)  =  n-'$y^'(nt).   In  addition,  note 
that 

t  <^lp\i)  <t+         max         Uk{i)/\[^\ 


hence  by  Lemma  3. .3  of  Iglehart  and  Whitt  [15], 


$^  —  e  u.o.c. 


7,13) 


We  begin  with  tasks  k  G  ^°,  for  which 

^^,{t)     =     .1^(0     and 

It  follows  from  (7.13)  and  Theorem  6.1  that  ^^^  —  e  u.o.c.   and  TJ'^.  —  T'^  where  T^^  =  W*^^^{t] 
The  theorem  is  then  proved  by  applying  induction  on  (3.15)-(3.16).  I 


8     Concluding  Remark,s 

We  presented  in  this  paper  a  heavy  traffic  analysis  of  feedforward  fork-join  networks  with  heteroge- 
neous customers.  We  made  several  assumptions  to  simplify  the  exposition,  but  the  results  proved 
here  apply  for  more  general  networks  as  well.  For  example,  we  assumed  that  each  station  is  staffed  a 
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single  server.  Using  the  machinery  developed  by  Chen  and  Shanthikumar  [7],  one  can  extend  these 
results  to  fork-join  networks  of  multi-server  queues.  Secondly,  whereas  we  assumed  that  all  servers 
are  reliable,  it  is  possible  to  analyze  networks  in  which  stations  may  experience  server  breakdown 
[13,  8].  Lastly,  batch  arrivals  can  be  accommodated  within  the  framework  presented  here  [20].  (The 
model  discussed  by  Baccelli  and  Liu  [4]  is  an  example  of  such  networks  would  thus  become  a  special 
case  of  Example  2).  In  this  case,  the  issue  reduces  to  calculating  Q,  the  covariance  matrix  of  the 
total  workload  input  process  [20]. 
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